Rewards Analytics and Distribution Dashboard

This goal of this notebook is to offer an easy way to process the outputs of the praise and sourcecred reward systems, perform an analysis of the results and calculate the token reward distribution. It uses mock data and should be considered a work-in-progress.

Imports

First, we import the relevant libraries, get the Data and set how many tokens we want to distribute

Now that we have selected the files, we can import them for processing. We can also set the total amount of tokens we want to distribute this period. We also set the name we want our output file to have.

Tip: Now that the file paths are set, you can safely click on "Cell > Run all below" from here on the menu bar to execute everything :)

Parameters

Set the total amount of tokens we want to distribute this period. We also set the name we want our output file to have.

Results Analysis

Let's dive into some data analysis! We'll use the metrics designed and explained by octopus🐙, and focus only on the praise rewards for now. Starting with:

Allocation percentages - Praise receivers

This table will show us which percentage of the total rewards gets distributed to which top % of users. So "Top 50% -> 0.85" would mean that the top 50% of praisees received 85% of the total rewards

	Rewards
Top 50%	0.956954
Top 20%	0.716975
Top 10%	0.490005
Top 5%	0.286515
Top 1%	0.061237

Allocation Percentages - Praise givers

This table will show us which percentage of the total rewards get distributed based on praise given by which top % of users. So "Top 50% -> 0.85" would mean that 85% of the total rewards are based on praise given by the top 50% praise givers.

	Rewards
Top 50%	0.958646
Top 20%	0.845796
Top 10%	0.719090
Top 5%	0.606346
Top 1%	0.485246

Plotting praise given against praise received

Now we can compare the relationship between praise given by a user and the praise they received

Gini coefficient

Next we will look at the Gini coefficient. Note that there is some debate if we want to use this metric at all, since it is usually employed to measure wealth distribution, and not compensation.

	Rewards
All	0.682305
Top 50%	0.458994
Top 20%	0.234393

Shannon Entropy

Shannon Entropy) is a concept from communications theory, which is also used in measuring the diversity of a distribution. The formula for calculating Shannon Entropy among $n$ individuals is $$\\sum_{k=1}^n -p_k log_2(p_k),$$ where $p_k$ represents the proportion of the resource that user $k$ received.

Here we compare the actual Shannon Entropy with the maximum possible for the dataset, keeping in mind that a Shannon Entropy of 0 would mean one user holds all the rewards

	Rewards
Entropy	5.380865
Max Entropy	6.599913
% of Max	0.815293

Nakamoto Coefficient

Last but not least, the Nakamoto coefficient. The Nakamato Coefficient is defined as the smallest number of accounts who control at least 50% of the resource. Although its significance relates to the prospect of a 51% attack on a network, which may not be relevant in our context, we can still use it as an intuitive measure of how many individuals received the majority of a resource.

11

Praise Data Visualization

Rating distribution

Since praise gets valued on a scale, we can take a look at how often each value of the scale gets assigned by quantifiers.

Praise Reward Distribution

We can also take a look at the distribution of the received praise rewards

Praise Outliers

To aid the revision process, we can generate a table which sorts the praise by the size of the spread between the highest and lowest scores. These "controversial" praise instances can then be discussed in the quantifier review call. Index numbers are kept to make localization in the "big praise sheet" easier.

	index	ID	DATE	TO USER ACCOUNT	FROM USER ACCOUNT	REASON	SCORE 1	SCORE 2	SCORE 3	DUPLICATE ID 1	DUPLICATE ID 2	DUPLICATE ID 3	DISMISSED 1	DISMISSED 2	DISMISSED 3	QUANTIFIER 1 USERNAME	QUANTIFIER 2 USERNAME	QUANTIFIER 3 USERNAME	AVG SCORE	SPREAD
0	326	621f79e243b89009366c8922	2021-08-12T22:00:00.000Z	Dan Knobelsdorf#326	iviangita#3204	for handling the Trusted Seed and because he k...	3	144	0	None	None	621f79e243b89009366c88e6	False	False	False	teaci#6108	Vyvy-vi#5040	divine_comedian#5493	50	144
1	189	621f79e143b89009366c880e	2021-08-05T22:00:00.000Z	sem 🐝#0161	Mount Manu#3530	for coming up with the idea of the TEC Hatch B...	3	144	8	None	None	None	False	False	False	PhilH#1102	Mount Manu#3530	calazki#3961	51	141
2	788	621f79e243b89009366c8b46	2021-08-23T22:00:00.000Z	🐙 octopus#5508	liviade#1387	for being so generous with his time, for being...	144	8	13	None	None	None	False	False	False	innov8tor3#3988	Mount Manu#3530	Mettodo#7010	55	136
3	831	621f79e243b89009366c8b7c	2021-08-26T22:00:00.000Z	Zeptimus#3359	iviangita#3204	for taking on the One-Time Change of TECH Addr...	21	144	8	None	None	None	False	False	False	AmwFund#0979	Mount Manu#3530	calazki#3961	57	136
4	290	621f79e243b89009366c88fc	2021-08-11T22:00:00.000Z	Vitor Marthendal#0090	karmaticacid#1218	for taking on project management position, for...	34	144	13	None	None	None	False	False	False	teaci#6108	Vyvy-vi#5040	kristofer#1475	63	131
5	252	621f79e143b89009366c88c1	2021-08-09T22:00:00.000Z	shawn#7046	iviangita#3204	for reaching out to translate some TEC article...	13	144	34	None	None	None	False	False	False	osycross(🦎🦎)#5943	Parrachia#0450	Usua◎ Silver#2618	63	131
6	581	621f79e243b89009366c8a38	2021-08-19T22:00:00.000Z	Vitor Marthendal#0090	iviangita#3204	for all the work on the CCD, the designs, comm...	13	144	34	None	None	None	False	False	False	teaci#6108	Vyvy-vi#5040	kristofer#1475	63	131
7	39	621f79e143b89009366c8774	2021-08-02T22:00:00.000Z	Zeptimus#3359	iviangita#3204	for creating the Hatch tutorial and bridge vid...	21	144	21	None	None	None	False	False	False	AmwFund#0979	Mount Manu#3530	calazki#3961	62	123
8	210	621f79e143b89009366c882b	2021-08-05T22:00:00.000Z	Vitor Marthendal#0090	markop#2007	for helping with the ABC Module design of the ...	13	89	1	None	None	621f79e143b89009366c87e3	False	False	False	teaci#6108	Vyvy-vi#5040	kristofer#1475	35	88
9	185	621f79e143b89009366c880a	2021-08-05T22:00:00.000Z	divine_comedian#5493	Mount Manu#3530	for slaying with the forum posts ✍🏻	1	5	89	None	None	None	False	False	False	PhilH#1102	acidlazzer#5796	calazki#3961	31	88
10	924	621f79e243b89009366c8bd9	2021-08-26T22:00:00.000Z	Vitor Marthendal#0090	iviangita#3204	for working on the params and there’s tons of ...	34	89	3	None	None	None	False	False	False	teaci#6108	Vyvy-vi#5040	kristofer#1475	42	86
11	225	621f79e143b89009366c883a	2021-08-05T22:00:00.000Z	chuygarcia.eth#6692	iviangita#3204	for the TEC Unconference Aftermovie and all th...	5	89	34	None	None	None	False	False	False	eduardovegap#0422	Zeptimus#3359	Usua◎ Silver#2618	42	84
12	494	621f79e243b89009366c89e1	2021-08-16T22:00:00.000Z	pKrepe#5682	griff (💜, 💜)#8888	for diving into the 4-hour hack sesh on sunday...	8	89	21	None	None	None	False	False	False	Sha#6179	acidlazzer#5796	bends#3537	39	81
13	1104	621f79e243b89009366c8ca0	2021-08-30T22:00:00.000Z	hernandoagf#3014	Jolie_Ze#0295	for for his consistent efforts to help me adap...	89	13	34	None	None	None	False	False	False	innov8tor3#3988	Mount Manu#3530	Usua◎ Silver#2618	45	76
14	310	621f79e243b89009366c8912	2021-08-12T22:00:00.000Z	BK#7471	iviangita#3204	for hosting the TEC Labs for 2 weeks while YGG...	13	89	55	None	None	None	False	False	False	PhilH#1102	Mount Manu#3530	calazki#3961	52	76
15	492	621f79e243b89009366c89df	2021-08-16T22:00:00.000Z	karmaticacid#1218	griff (💜, 💜)#8888	for diving into the 4-hour hack sesh on sunday...	34	89	13	None	None	None	False	False	False	innov8tor3#3988	acidlazzer#5796	bends#3537	45	76
16	840	621f79e243b89009366c8b85	2021-08-26T22:00:00.000Z	elessar.eth#7945	iviangita#3204	for the work on EVM-crispr	21	89	89	None	None	None	False	False	False	PhilH#1102	Vyvy-vi#5040	Usua◎ Silver#2618	66	68
17	829	621f79e243b89009366c8b7a	2021-08-26T22:00:00.000Z	liviade#1387	iviangita#3204	for taking a week of vacay	0	0	55	None	621f79e243b89009366c8b65	None	True	False	False	Sha#6179	tigeroid#2951	Mettodo#7010	27	55
18	277	621f79e243b89009366c88ef	2021-08-11T22:00:00.000Z	Dan Knobelsdorf#326	karmaticacid#1218	for helping sort through and support people in...	5	55	0	None	None	621f79e243b89009366c88e6	False	False	False	teaci#6108	Vyvy-vi#5040	divine_comedian#5493	21	55
19	580	621f79e243b89009366c8a37	2021-08-19T22:00:00.000Z	markop#2007	iviangita#3204	for all the work on the CCD, the designs, comm...	0	55	55	621f79e243b89009366c8965	None	None	False	False	False	AmwFund#0979	Parrachia#0450	Usua◎ Silver#2618	36	55
20	630	621f79e243b89009366c8a69	2021-08-19T22:00:00.000Z	iviangita#3204	Mount Manu#3530	for the really great work on Twitter, engaging...	55	21	0	None	None	621f79e143b89009366c87bd	False	False	False	innov8tor3#3988	Zeptimus#3359	enti#1546	25	55
21	796	621f79e243b89009366c8b4e	2021-08-24T22:00:00.000Z	natesuits#4789	griff (💜, 💜)#8888	for coming to the Params hack sesh on sunday!	5	1	55	None	None	None	False	False	False	innov8tor3#3988	Parrachia#0450	calazki#3961	20	54
22	588	621f79e243b89009366c8a3f	2021-08-19T22:00:00.000Z	karmaticacid#1218	iviangita#3204	for sharing the Giveth Covenant with Livia	55	8	1	None	None	None	False	False	False	innov8tor3#3988	acidlazzer#5796	bends#3537	21	54
23	930	621f79e243b89009366c8bdf	2021-08-26T22:00:00.000Z	chuygarcia.eth#6692	iviangita#3204	for jumping back and leading comms	1	55	21	None	None	None	False	False	False	eduardovegap#0422	Zeptimus#3359	Usua◎ Silver#2618	25	54
24	66	621f79e143b89009366c8793	2021-08-03T22:00:00.000Z	Zeptimus#3359	liviade#1387	for making hatch tutorials!!	2	55	2	621f79e143b89009366c8774	None	621f79e143b89009366c8774	False	False	False	AmwFund#0979	Mount Manu#3530	calazki#3961	19	53

We can also plot the spread against the average score for a more visual search.
ATTENTION! Keep in mind that this is just meant as a visual aid. If there are several praise instances with similar spread and average quant number, all but one end up "hidden" on the chart. For the full list you have to use the table.

Praise Flows

Now for something more fun: let's surface the top "praise flows" from the data. Thanks to @inventandchill for this awesome visualization! On one side we have the top 20 praise givers separately (modifiable by changing the variable n_senders), on the other the top 25 receivers (modifiable by changing the variable n_receivers). The people outside the selection get aggregated into the "REST FROM" and "REST TO" categories.

SourceCred Data

SourceCred token Distribution

Next we can see the distribution made by the SourceCred algorithm.

Quantifier Data

Let's take a closer look at each quantifier. In the following step we will use the raw praise data to zoom in on how each quantifier scored the praises:

Amount of praise quantified

With the above table we can easily see how much praise each quantifier rated.

Mean praise score by quantifier

Let's see the mean praise scores every quantifier gave.

Total Token Distribution Visualization and Export

To send the allocations to the DAO for distribution, we need to put all data together and add the rewards for the reward board and the quantifiers.

Let's take a final look at the total distribution. Click on the legend to filter out specific reward sources

That's it! We can now download the final distribution files and upload them to GitHub for future reference. We want to export 3 different files:
 - The final reward allocations, separated by source
 - The final reward allocations, in a disperse.app-readable format
 - The extended praise data, detailing the how many tokens each single praise netted (for future reference)